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ABSTRACT 


The  class  of  Interpolatory-Newton  iterations  is  defined  and  analyzed 
for  the  computation  of  a simple  zero  of  a non-linear  operator  in  a Banach 
space  of  finite  or  infinite  dimension.  Convergence  of  the  class  is  estab- 
lished. 

The  concepts  of  "informationally  optimal  class  of  algorithms"  and 
"optimal  algorithm"  are  formalized.  For  the  multivariate  case,  the  optimal- 
ity of  Newton  iteration  is  established  in  the  class  of  one-point  iterations 
under  an  "equal  cost  assumption  ". 


.1.  INTRODUCTION 


In  Traub  and  Wozniakowski  [76c]  we  investigate  the  class  of  direct  inter- 
polator iterations  I for  a simple  zero  of  a non-linear  operator  in  a Banach 
space  of  finite  or  infinite  dimension.  The  solution  of  a "polynomial  operator 
equation"  is  required  at  each  step.  In  .this  paper  we  consider  the  solution 
of  this  polynomial  operator  equation  by  a certain  number  of  Newton  iteration 

steps.  We  call  this  the  class  of  Interpolatory-Newton  iterations  IN  . We 

n 

analyze  the  convergence  and  complexity  of  this  class. 

Traub  and  Wozniakowski  [76c]  show  that  the  radius  of  the  ball  of  conver- 
gence of  I can  grow  with  n.  Since  IN  uses  Newton  iteration  as  its  "inner 
n n 

process"  its  convergence  characteristics  are  similar  to  Newton  iteration 
(Traub  and  Wozniakowski  [ 77  ])  and  the  convergence  is  only  "local".  A "type 
of  global  convergence"  is  established  for  a certain  class  of  operators. 

The  complexity  analysis  of  IN^  requires  some  new  complexity  concepts.  We 
formalize  the  idea  of  "optimal  algorithm".  Under  an  "equal  cost  assumption" 

(and  one  additional  reasonable  assumption)  we  establish  the  optimality  of 
for  scalar  problems  and  the  optimality  of  IN2  = I 2 (Newton  iteration)  for 
multivariate  problems.  However,  if  the  equal  cost  assumption  is  violated  a 
high  order  iteration  is  optimal. 

We  summarize  the  results  of  this  paper.  Convergence  of  the  class  of 
iterations  is  established  in  Section  3.  General  complexity  results  are  obtained 
in  Section  4 and  used  to  establish  the  optimality  results  of  Section  5.  In  the 
final  section  we  analyze  a class  of  problems  for  which  Newton  iteration  is  not 
optimal . 
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2.  INTER POLATORY- NEWTON  ITERATION  IN 

n 

In  Traub  and  Wozniakowski  [76c]  we  consider  interpolator  iteration  I 

n 

for  the  solution  of  the  non-linear  operator  equation 

(2.1)  F (x)  - 0 

where  F:  D c -•  and  B^,  are  real  or  complex  Banach  spaces  of  dimension 
N»  N ■ dim(B.)  ■ dim(B_) , 1 ^ N s +00.  The  interpolatory  iteration  I is  de- 
fined  as  follows.  Let  xt  be  an  approximation  to  the  simple  solution  a and  let 
be  the  interpolatory  polynomial  of  degree  s n-1  such  that 

(2.2)  w£J)(Xl)  -F(j)(Xi),  j - 0,1,. ...n-l 

where  n ^ 2.  The  next  approximation  *1+1  is  a zero  of  w^  wi(*1+1)  “ 0,  with 
a certain  criterion  of  its  choice.  Note  that  for  n * 2 we  get  Newton  itera- 
tion since  x^+j  " xi  ” ^ (x^) . The  degree  of  w^  is,  in  general,  equal 

to  n-l  and  for  n s 3 we  get  a "polynomial  operator  equation"  for  x^*^ . There 
are  a number  of  ways  for  dealing  with  the  problem  of  solving  this  equation 
numerically.  In  this  paper  we  will  approximate  x1+1  by  applying  a number  of 
Newton  iterations  to  the  equation  w^x)  - 0.  Let 

Z ■ X 

0 i 

(2.3)  2j+1  - Zj  - w'(lj)"Iw1(«J)t  J - 0.1 k-1 

xi+l  “ zk 

where  k - riog2  nl . 

We  shall  call  the  iteration  constructing  {x^  by  (2.3)  the  interpolatory- 
Newton  iteration  INn.  This  is  a one-point  stationary  iteration  without  memory 
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in  the  sense  of  Traub  [64],  Note  that  for  n = 2,  (2.3)  reduces  to  Newton 
iteration  and  ” Ij*  To  compute  zj+^  006  must  solve  the  linear  equation 
w'(Zj)  " w^(zj)*  We  do  not  specify  what  algorithm  is  used  to 

solve  this  linear  equation.  In  fact,  IN^  is  the  name  of  a class  of  iterations 
which  use  the  same  information  (2.2),  and  perform  k Newton  steps  to  solve 
w^(x)  ■ 0 but  they  can  differ  in  the  algorithm  used  to  solve  the  linear  equa- 
ton.  For  example,  by  Newton  iteration  we  mean  any  iteration  which  produces 


i+1 


- F'  (x^  *F (x^)  no  matter  what  algorithm -is  used  to  compute  x^+^. 


Sone  properties  of  the  interpolatory  iteration  I , n ^ 2,  will  be  used 
to  establish  the  convergence  of  INn»  Let  a be  the  simple  solution  of  F(x)  ® 0 


and  J * [x:  | (k- or 1 1 £ f).  Define 


(2.4)  A.  - A (H  - sup  | )F ' ( or) ” 1 F |t^||»  j * 2,3, 
j J x€J 


whenever  F^  exists. 

From  theorem  2.1  in  Traub  and  Wozniakowski  [76c]  directly  follows 


Theorem  2.1 


If  F is  twice  differentiable  in  J and 


(2.5)  A2rs4, 


(2.6)  xt  € J 


then  the  next  approximation  xt+1  constructed  by  Newton  iteration  satisfies 


(2.7) 


-a  - |F,(a)‘1F"(0f)(xt-a)2  + o(  ) . 


i+1 


T 
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3.  CONVERGENCE  OF  INTERPOLATOR Y- NEWTON  ITERATION 


We  study  the  convergence  of  IN  for  n * 3.  Let  e,  * |tc  -a|  , v . 

n l Ir  i 1 ’ i 


Theorem  3.1 


If  F is  n-times  differentiable,  n 2 3,  in  J and 


(3.1)  O^fsl/5 


where 


A,  + SiSLli*  (2D-2 

z z n 

. n- 1 ’ 

1 ■ v - ^Xl)  ^ 


(3.2)  xQ  6 J 

then  the  sequence  {x,  } constructed  by  the  interpo la tory- Newton  iteration  IN 
is  well-defined  and 


(3.3)  € J,  Vi, 


(3.4)  lj.xl-flr,  ei+l  S tl  + Kj)  K’  Vi* 


(3.5)  e s C e where 
It  1 1 , n 1 


"ic  "it 

for  ®i+l  " ^i+l  Hi  " 0(eL)  and  0 SH1  k-  riog2  nl , 


n-1 


lim  C^  „ ■ An  + 6A2  where  8 ■ 0 if  2 ■ n,  and  6 ■ 1 if  2*  * n, 


T 


J 
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(3,6)  Xi+r°'  " Fn(V“)n  + bi,k  + o(lllci"a^) 


where 


bi,l  “F2(xi-o)' 


bi,j+l  = F2  bi,j  * j 3 


F - F’  (a)_1F(j)(a)  for  j = 2 and  n. 

J J • 


Proof 


Assume  by  induction  that  x^  € J.  We  want  to  show  that  the  interpolatory- 
Newton  iteration  is  well-defined,  i.e.,  w^(z^)  is  invertible.  First  we  shall 
prove  that  w1 (x)  is  invertible  for  x € J and  next  that  z € J.  Denote 

(3.7)  F^(x)  - w^  (x)  ■ R^  (xjXj)  for  x 5 J,  j = 0,1,2, 


where 


|^(a)_Vj)(x;xi)||  * jiQAjlx-xjp-J, 


see  Rail  [69,  p.124].  Since 


w^(x)  - F'  (x)  - R^(x;xt)  * F'  (or)  [ I + F'  ( or) “ 1 {F ’ (x)  - F'  ( a ) } - F'  (o)“  V (x;x  ) ] 


then  from  (3.7)  and  (3.1)  we  get  for  x € J 


(3.8)  |^'(a)'1w'(x)  - l||  * 2A2|k-fy||+  nAjk-xJf1  s 

« 2A„r  + nA  (2T)n_1  S 4 < 1. 

I n 5 


From  theorem  10.1  in  Rail  [69,  p.36]  it  follows  that  w^(x)  is  invertible  for 


any  x € J and 
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(3.9)  |^(x)“V(») 


- 2A2llx-a||  -nAn|(x-xi|P* 


Since  the  denominator  in  (3.1)  is  positive  then 


v_  <(i? 

1-A V3  / 


and  from  theorem  2.2  follows  that  the  polynomial  has  a unique  zero  in 
Jx  - {x:  |(x-aj|  Sjr],  w(x1+1)  = °»  and  (2.10)  holds.  From  (3.9)  and  (3.7)  we 
get  for  x € J 


w**  (x) 

(3.10)  |^(xi*1)'1  II  s |^(xi*1fV(a)||  |^(a)' 


^(x) 


s A2  + ^ A2  + ~^2~1)An(2r)n~2 

1 • 2A2ltxiIrQrH  ' ^nlK+l"*!^"1  1 - A2r  " "Vi0”"1  ^ 

We  investigate  the  properties  of  (z . } defined  in  (2.3).  Recall  we  solve 
wi(x)  - 0 by  Newton  iteration.  Note  that  - F' (x.j'Vx.)  is  the 

Newton  step  applied  to  the  equation  F(x)  - 0.  From  (3.1)  we  know  that 
A2r  s 1/ 4 and  from  theorem  2.1  we  get 

Since  xi*1  € Jy  1^-xJji  £ T. 

We  prove  that  zj+1  lies  in  Dj  - (x:  l^^ll  * } 0 J.  Let 

(3.11)  wt(x)  - w1(Zj)  + w'(zj)(x-zj)  + R2(x;z.) 
where  R2(x;y)  - J*  w^(y+t(x-y) ) (x-y)2 (l-t)dt, 


A + -^~1}A  (2Dn‘2 
2 2 n 
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for  x,y  € J,  compare  with  (3.7).  Note  that  zj+1  is  the  zero  of  the  equation 

(3.12)  x =*  H(x)  — x.^  + w' (xi+1)"1fR2(x;zj)  - ^(xjx.^)}. 

We  show  that  H is  contractive  on  D..  From  (3.10)  we  have  for  x f D. 

^ j 


S^dlx-Zjll2  4-  Itx-x.^ll2)  sllfz.-, 


i+1 1 


due  to  (3.1).  Furthermore 


|(H(x)-or||  * |txi*1-cv||+||H(x)-xi*1||  s (|  + I)r=  r. 


Thus  H(D j)  — Dj.  Since  |^'  (x)  ||  5 2A^  ||z ^-x  < 1 then  H is  contractive 

on  D . and  z . , . is  the  unique  zero  in  D . 

J J+l  J 

This  proves  that  xi+1  = € J and 


IK+r®!! * |txi+rxi+iH + IK+HI  *(i)  lhrxi+ill  + iHv®! 

S(|G)  + i)  ^i'°'H  G l^i-"ll 

which  yields  (3.3)  and  (3.4). 

Let  ej  - llzj'xi+iil  • Set  x = z .+l  in  (3.12).  Then 


(3-13)  Vi s ~ >/ej>  *5  s ¥“>'?! 

1 " Vj+i 


where  ^ ■ O(e’j)  and  0 s Hi  S5/2,  compare  (2.7).  Since  e\ 
write  ^ * CKe^)  . Next  from  (3.13)  and  (2.10)  we  get 


O(e^)  we  can 
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!i+i ' ll*i+r“ll s ^i+rxiIiH+  ll«i+r«ll  ■ \ 


, t~2(l+H  > ]2-1lk1-xi:ili2fc  ♦ — ^-r(x*  ‘Al  « 

1 - Vi+I V ^ 


* Nn/ 

e , . A / A 


* 1 + 


*V  2-n 


cuA 


/ 

Since  e^j/ tend  to  zero  then 


lim  C.  =»  A +6  A„n-i 
i > n n 2. 


where  5 =*  0 if  2 > n and  6*0  otherwise.  Hence  (3.5)  holds. 


Finally  observe  that 


★ 

. v**  (x  ) * 

zj+l  - xi+l  “ wl(xi+i>  — (2j-xi+i^  + °^> 


F2(Zj-xi*i)2  + o(^). 


\ " xi+i  " f2(f2* • • • ‘ (F2^Xi“Xi+l^  ^ •**  ^ + °(ei  ) 


F,(F,>../(F,(x-9)2)2  ...)2  + o(e2  ) 


From  the  definition  of  b.  in  (3.6)  we  get 

1 , R 


zk  - xi*i  " bi,k  + °(ei  >■ 
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From  this  and  (2.11)  we  have 


x...  - a = z - x + x...  - <y  » b.  + F (x.-a)n  + o(en) 
l+l  k l+l  i+l  i,k  nv  l v l 


which  proves  (3.6)  and  completes  the  proof  of  theorem  3.1. 


Remark  3 . 1 

It  is  possible  to  get  a slightly  better  estimate  than  (3.1)  although  the 

proof  is  much  more  complicated.  Note  that  if  n is  not  a power  of  2 then 

k 2k 

2 > n and  the  leading  term  in  (3.6)  is  F (x.-a)  since  ||b.  ||  = 0(e.  ) = o(e  ) . a 

n l l ) K l i 

Remark  3.2 

The  idea  of  using  Newton  iteration  to  estimate  a zero  of  an  approximating 
non-linear  operator  which  fits  the  information  of  F can  be  applied  for  any 
iterations  with  or  without  memory;  see  Brent  [76]  where  Newton  iteration  is 
also  used  as  an  inner  iteration.  ® 


Remark  3 . 3 

Since  Newton  iteration  is  numerically  stable  it  is  relatively  easy  to 

verify  the  numerical  stability  of  the  interpolatory-Newton  iteration  IN  under 

n 

appropriate  assumptions  on  the  computed  information  of  F,  see  Wozniakowski 
[76b].  » 


In  general  the  interpolatory-Newton  iteration  converges  only  locally.  We 

give  conditions  under  which  IN^  enjoys  a "type  of  global  convergence".  Compare 

the  same  property^ or  Newton  iteration  in  Traub  and  Wozniakowski  [ 77].) 

Let  F(x)  * ^ yY  (»)  (x^or)  L be  analytic  in  D * {x : |(x-a||  < r]  and 
i=l 

UF'(«r1F(i)(ey)ii  i-i 
i*.  K 


(3.14) 
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for  i ■ 2,3 R 2 1/k. 

One  way  to  find  K is  to  use  Cauchy's  formula 


•.’(or)'1T(1)(a)  ||  M 


where  M - sup  |£'  (or)  F (x)  ||  . Setting  K - max(-i  we  get  u/R  SRR  s (KR)1"1 


i i 

which  yields  M/R  £ K . 


R*  r2> 


Theorem  3.2 


If  F satisfies  (3.14)  then  the  interpolatory  Newton  converges  for 
Xg  € J * [x:  | ^c- orj | s T"  } where 


n K 


and  xq,  0 < < x#,  satisfies  the  equation 


(3.15)  5 


(1-x)- 


n(n-l)  / 2x  V 
4 (1-x)2  V"x/ 


1 x n / 3x 

' (1-x)3  ' (l-xA2*1-*) 


and  xn  ' xo  where  x^  is  the  smallest  positive  solution  of  x/(l-x)3  =*  1/6  and 


x * .12. 


Proof 


From  (3.14)  we  get 


|f'(*r1F(i)(x)||  S f(i)(||x-a||  ) 

where  f(x)  - x/(l-Kx).  Since  f(i)(x)  - i'.R1"  V (1-Kx) 1+1  for  i 2 2 , we  have 


Ai(F)  * TIT  » 1 " 2»3 

(1-KH  1+i 
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Then  (3.1)  becomes 


*asa^%£r)/(- 

4(1-KD  V1  K / / V 


k r 


(l-KO' 


(l-KO* 


(-  \ 

\2(1-K  T)J 


l 

5 


Setting  Kf  = x we  get  that  x satisfies  the  equation  (3.15).  It  is  straight- 
forward to  verify  that  x ■ x(n)  is  an  increasing  function  of  n and  lim  x(n)  = x 

00 

n 

where  xffl  satisfies  the  equation 


x 

(l-x)3 


1 

6 * 


Hence  x^  = .12  which  proves  theorem  3.2. 

This  result  is  especially  interesting  if  the  domain  radius  R is  related 
l x x 

t0  K’  Sa^  ^ * K~ * T^1611  = 3 ~ R — ~ — R and  the  interpolatory-Newton 

iteration  enjoys  a "type  of  global  convergence". 
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4.  COMPLEXITY  OF  ONE- POINT  ITERATIONS 

In  this  section  we  deal  with  complexity  of  one-point  iterations.  We 


extend  some  of  the  results  of  Traub  and  Wozniakowski  [76a]. 

Assume  that  a one-point  iteration  cp  constructs  the  sequence  x 
converging  to  a and  satisfying 


'P(xi;F) 


(4.1)  et  =*  Gi  e±  » H^-all,  i = 1,2,  ...,K 

where  p,  p > 1,  is  called  the  order  of  iteration  cp, 

(4.2)  0 < G * G.  S g < +®,  i = 1,2, ...  ,K 

and  the  iteration  is  terminated  after  K steps. 

From  (4.1)  we  get 


(4.3) 


e 


0* 


Note  that  (e^)  P is  the  geometric  mean  of  the  Gj^  ,G2 , . . . .G^  Furthermore 
ei  < eQ  iff  > 1.  From  (4.2)  we  get 


(4.4) 


1 

w 


1 

w 


We  shall  assume  that  w > 1. 

For  a given  c' , 0 < e'  < 1,  let  K be  the  smallest  index  for  which  e £ c'e  . 

k 0 

Define  e < e'  so  that 


(4.5)  eR 


ee 


0* 
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Let  comp  - comp(co,F)  be  the  total  cost  of  finding  x^,.  Assume  that  the 
cost  of  the  ith  iterative  step  does  not  depend  on  the  index  i;  we  denote  it 
by  c = c(a;F) . Then 

(4.6)  comp  ® cK. 

From  (4.3)  and  (4.5)  we  get 


where 


g(wR) 
log  p 


(4.7)  g(w)  =*  lg(l  + -j^),  t =*  lg  1/e. 

We  take  all  logarithms  for  the  remainder  of  this  paper  to  base  2.  Then  from 
(4.6)  and  (4.7)  we  get 


(4.8)  comp  * z g(wR) 
where 


(4.9) 


z 


c 

Igp 


is  called  the  complexity  index. 

Since  g(w)  is  a monotonically  decreasing  function,  (4.4)  gives  bounds  on 
complexity 


(4.10)  zg(w)  £ comp  £ zg (w) . 


Note  that  as  e -•  0,  g(w)  2;  Igt  and  comp  ^ zlgt.  If  we  assume  that 
(4 . 11)  2 s w s w s t 
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then  (4 . 10)  becomes 


(4.12)  z(lgt-lglgt)  s comp  £ z lg(l+c); 


see  theorem  3.1  In  Traub  and  Wozniakowski  [76a].  In  this  case  the  complexity 
index  is  a good  measure  of  complexity. 

We  want  to  minimize  the  total  cost  of  finding  x . More  precisely,  for 

tv 

a given  operator  F we  want  to  find  an  iteration  5 with  minimal  complexity. 

Since  we  do  not  know  the  value  g(w  ) in  (4.8)  we  are  not  able  to  minimize 

1\ 

complexity.  However,  if  (4.11)  holds  or  e is  small  enough  then  minimal  com- 
plexity is  approximated  for  an  iteration  with  minimal  complexity  index.  So 
we  wish  to  find  an  iteration  $ which  for  a given  problem  has  as  small  a com- 
plexity index  as  possible. 

The  complexity  index  is  given  by  z ■ c/lg  p where  c is  the  cost  per  one 

iterative  step  and  p is  the  order  of  an  iteration.  Assume  that  an  iteration 

cp  uses  the  standard  information  (see  Wozniakowski  f 75a])  51  =51  (x:F)  ■ 

' n L n n 

[F  (x)  ,F' (x)  , . . . ,F^n  ^(x)}.  The  cost  c • c(cp  ,51  ) consists  of  the  information 

n n 

complexity  u ■ u(F,5tn)  which  is  the  cost  of  computing  5l^(x;F)  and  the  com- 
binatory complexity  d ■ d(cpn)  which  is  the  cost  of  combining  information  and 
producing  thf  next  approximation.  Then  c ■ u+d  and  (4.9)  can  be  rewritten  as 


(4.13)  z(®n) 


u(F,51q)  + d(cpn) 

lg  P(®n) 


We  want  to  find  an  iteration  ® which  minimizes  (4.13).  Let 

n 


(4.14)  z (F)  = inf  z ( cd  ) , 

« €* 

n n 


(4.15)  z(F)  - inf  z (F) , 
n*2 
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where  is  the  class  of  one-point  iterations  using  the  standard  informa- 
tion with  n evaluations.  (Of  course  IN^  € 5 , Vn.)  We  need  lower  and  upper 
bounds  on  zn(F)  and  z(F) . Note  that  upper  bounds  can  be  obtained  by  the  com- 
plexity index  of  any  iteration  ®n>  e.g.,  by  the  complexity  index  of  the  inter- 

polatory-Newton  iteration  IN  . We  shall  deal  with  this  in  Section  5. 

n 

Throughout  the  rest  of  this  paper  we  assume  that  the  dimension  of  the 
problem  is  finite,  N < +°°. 

To  find  lower  bounds  on  z (F)  and  z(F)  we  need  a lower  bound  on  z(cp  ) . 

n Tn 

Note  that 


h 


(4.16)  u(F;^n)  = u c (F (i) ) 

i-0 


where  c(¥^^)  denotes  the  "cost"  of  computing  F^^  (x) . In  the  "cost"  one  can 
include  all  the  costs  of  computing  F^  including  the  cost  of  all  arithmetic 
operations  needed  to  compute  F^,  the  cost  of  variable  data  access,  the  cost 
of  subroutine  calls,  etc.  For  the  sake  of  simplicity  we  assume  that  the  cost 
of  one  arithmetic  operation  is  taken  as  unity. 


In  general  F^(x)  requires 


»tr) 


different  data  for  its  representation. 


The  total  number  of  data  in  H is  equal  to 

n 


(4.17)  d 


n-l  . N . ^ 

. „ (w-L 

N,n  i—  j/  V n-l  J 

j-o 


For  almost  all  problems  the  information  cost  u(F;Uln)  depends  linearly  on  d^ 
To  make  this  more  precise  we  introduce 


Definition  4 . 1 (Functional  Independence  Assumption) 

We  say  F satisfies  the  functional  independence  assumption  if  there  exists 
a positive  constant  c^  such  that 
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(4.18)  u(F;30  a CF  dN  n»  Vh.  ■ 

If  F depends  on  all  different  data  in  31^  then,  of  course,  cp  is  at 

least  equal  to  unity.  However  it  can  happen  due  to  a special  property  of  F 

(like  symmetry  of  some  F^)  that  the  information  cost  u(F;3l  ) is  less  than 

n 

d . In  the  functional  independence  assumption  we  need  not  specify  the 
N ,n 

value  of  cp  as  long  as  cp  is  positive. 

We  estimate  the  combinatorial  complexity  d(cDn).  Any  iteration  cp  has  to 
use  every  piece  of  data  at  least  once  as  well  as  the  current  approximation  to 
or.  Thus 


(4.19)  4(9n>idN,n- 

Since  P(<Pn)  ^ n (see  Traub  and  Wozniakowski  [76b])  for  any  iteration  co  we  find 
u(F;3t  ) + d(o  ) d 

<‘-20)  ITTOT 2 (CF+1)  Ti^n  2 <V1)V1 


where  ^ ^ for  N * 1 and  * N(N+1)  for  N ^ 2.  Thus,  we  proved 
Theorem  4 ■ 1 

For  the  class  of  one-point  iterations  which  use  the  standard  information 
of  F with  n evaluations  where  F satisfies  the  functional  independence  assump- 
tion, the  minimal  complexity  indexes  zn(F)  and  z(F)  are  bounded  below  by 


(4.21)  z (F)  ^ (c  +1)  , 

n F lg  n 


(4.22)  z (F)  2 (cp+l)< 


lg  3 


for  N = 1 


N (N+l)  for  N 2 2, 


< 


5.  OPTTMALITY  THEOREMS 


In  this  section  we  deal  with  the  complexity  of  the  interpolatory- Newton 
iteration  IN^,  n S 2.  Recall  that  under  the  assumptions  of  theorem  2.1  for 
n “ 2 and  theorem  3.1  for  n a 3 the  iteration  INn  constructs  the  sequence 


{x^}  such  that 


(5.1)  e.-G.e^ 


where  e^^  - cr||  and  s G for 


\/d  - 2A20 


for  n = 2 


(5.2)  G * G (n) 


,k_ 
~ \2-  1 


•(1+q)n(^y72  + (K) 


?k\ 

[(l+q)rf'n)  for  n > 2 


where  A,,  is  defined  by  (3.1),  q 3 | + |(|)k  and  k = ri8  nl ; see  (2.7)  and  (3.5) 
Furthermore,  recall  (2.7)  and  (3.6), 

(5.3)  xi+1-or  - Fn(x.-o)n  + b.>k  + o(  ((x.-orlf1) 

where  3 (-1)  ~ F' (or)  *F^(a)  and  bi  k given  by  (3.6)  is  omitted  for  n = 2 

From  (5.3)  it  is  reasonable  to  assume  that 


(5.4)  ei+1*Ge" 


for  i * 0,1,... ,K, 


where  G 3 G(n)  is  a positive  number. 


Theorem  5.1 

If  G (2T-)n-1  * 1 then 

(5.5)  comp(INn)  s z(INn)  lg(l+t),  t 3 lg  j. 
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If  G (tnn_1  :>  1 then 


(5.6)  comp ( IN  ) 2 z(IN  ) (l gt- lglgt) . 
n n 


Proof 

From  (4.4),  (4.11)  and  (4.12)  it  is  enough  to  assure  that  » * jjl  (n-l)r  s _1 
1 . w 2 

and  — * G ^ ^ t This  is  equivalent  to  5(20°  ^ £ 1 and  G(tr)n-1  ^ 1 

which  hold  due  to  the  assumptions . ■ 


Remark  5 . 1 


To  assure  convergence  of  the  interpolatory-Newton  iteration  IN  , n a 3, 

n 

— 1 

we  assumed  in  theorem  3.1  A2"  5 y.  To  get  a good  upper  bound  on  complexity 
we  have  to  strengthen  this  inequality  to  G(2r)n-1  ^ 1.  It  may  be  shown 

■ ^ O 

that  G(2D  s 1 for  n = 2 implies  A2P  S yy  . (However  both  inequalities 
~ _ 1 ~ „ 2 

A2  4 j and  A2I  seem  to  be  slightly  overestimated.) 

Note  that  in  general  a stronger  condition  is  needed  to  get  "good  complexity" 
than  merely  to  assure  convergence.  In  Traub  and  Wozniakowski  [77  ] we  showed 
that  for  Newton  iteration  it  is  necessary  to  assume  < y for  convergence 
and  A2T  s jr  for  "good  complexity".  Note  that  for  n - 2 , G2I"  4 1 is  equivalent 
t°  a2t  *£.  n 

We  discuss  the  complexity  index  z(INn)  of  the  interpolatory-Newton  itera- 

I 

tion.  Recall  that  the  next  approximation  is  obtained  by  k * fig  nl  Newton 
steps  applied  to  w^(x)  *0.  It  may  be  shown  that  the  total  number  of  arith- 
metic operations  sufficient  to  perform  one  iterative  step  of  IN  is  equal  to 

n 


O^Wlg  nl  + (fig  nl  - 1)^ 


(5.7)  d( IN  ) 

IS-nflg  nl  + o(l) 


for  N 2 2 


for  N = 1 


* ■ ■■'» 
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where  the  total  number  of  arithmetic  operations  necessary  to  solve  a system 

fl 

of  N linear  equations  is  0(NK)  , j3  s 3. 


Example  5.1 

Consider  the  iteration  IN^  for  the  scalar  case.  We  solve  w. (x)  = 0 by 
applying  k = Tig  3l  = 2 Newton  steps.  Then 


F(x.)  F"(x.)  [F(x.)y  l 

Xi+1  “ Z2  * Xi  F' (x.)  ‘ 2F ' (x.)  F' (x.)  TTp. 

i i _ v i i 


F"(x^)  F(x^) 

where  T'Jx. ) ’ T*lus  the  combinatory  cost  d(IN.j)  = 9. 

Note  that  = O(e^)  . It  may  be  shown  that  p =»  0 does  not  affect  the 
order  of  iteration.  Therefore  one  can  define  the  next  approximation  x^_^  as 


_ _ F(x.)  F"(Xi)  p(xt) 

(5'8)  Xi+1  * Xi  " F' (x  ) ‘ 2F' (xt)  F' (x.) 


It  is  easy  to  show  that  x^+^  = w^(0)  where  w^J  (F(x^))  ■ gVJ/(F(x^))  for 
j * 0,1,2  where  w^  is  a polynomial  of  degree  at  most  2 and  g(x)  = F (x)  is 
the  inverse  function  of  F,  see  Kung  and  Traub  [74].  The  iteration  which 
constructs  x^+^  is  called  the  inverse  interpolatory  iteration  1^  and  d(I^)  = 7.  g 

A similar  upper  bound,  0(n  lg  n) , on  the  combinatory  complexity  for  the 
scalar  case  for  inverse  interpolatory  iteration  has  been  obtained  by  Brent 
and  Kung  [76]. 

We  believe  there  exist  no  iterations  with  essentially  less  combinatory 
cost  than  that  given  by  (5.7).  We  propose 
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Conjecture  5.1 

The  combinatory  complexity  d(®n)  of  any  iteration  with  maximal  order, 

p(o  ) = n,  has  to  be  at  least 
n 


c 


1 


/N+n-l\ 

\n_1/ 


log  n 


for  n ^ 3 


and 


for  n = 2 


for  positive  c^  and  c^  independent  of  n and  N. 


We  turn  to  the  problem  of  bounds  on  the  minimal  complexity  index  z (F) 

n 

and  z(F),  see  (4.14)  and  (4.15). 

Definition  5.1 
Let 

z(F)  - zn*(F) 

★ ★ 

for  some  integer  n . Then  we  say  n is  the  optimal  information  number  with 

respect  to  z(F)  (or  the  optimal  information  number)  and  is  the  optimal 

information  set  (among  one-point  standard  information)  . An  iteration  (j)^*  (or 

a class  of  iterations  d)  *)  is  said  to  be  informationally  optimal.  B 

n 

From  (4.14)  and  (4.15)  we  get 


z (F) 
n 


n- 1 

c(F(i))  + d(INn) 


i=0 


lg  n 


» 
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I 
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I 

i 

r 

b 


To  get  further  estimates  on  the  complexity  index  and  to  find  the  optimal 
value  of  n we  must  specify  a relation  among  the  c(F<^l)).  Let  c(F)  denote  the 
cost  of  evaluating  F (x)  and  assume  that  each  new  piece  of  data  in  F(x) ,F' (x) , . . . 
costs  the  same  number  of  arithmetic  operations.  For  most  problems  c(F)  is 
proportional  to  N. 


Definition  5.2  (Equal  cost  assumption) 

We  say  F satisfies  the  equal  cost  assumption  if 


(5.10)  c(F(l)) 


for 


1,2,... 


Note  that  the  equal  cost  assumption  implies  the  functional  independence  assump- 
tion but  that  the  converse  does  not  hold. 

From  (5.10)  we  get 


(5.11) 


n- 1 

1 c(F(i)) 

i-0 

lg  n 


NFn- 1) 
lg  n 


(N+l)  c (F) 


c(F)  * 


lg  3 


c(F) 


for  N ^ 2 
for  N = 1. 


Theorem  5.2 

If  F satisfies  the  equal  cost  assumption  then  the  minimal  complexity  index 
z(F)  satisfies 


(i)  for  N = 1 


(5.12)  ^ c(T)  + ^ * z(F>  - CCH  + 

(5.13)  z(F)  * z^(F)  whenever  c(F)  ^ = 23 
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* 


j 


which  means  that  the  optimal  information  number  n = 3 1 
(ii)  for  N 2 2 

(5.14)  (N+l)  [c(F)+N]  * z (F)  ^ (N4-1)[c(F)  + d(IN2)/ (N+l)  ] , 

(5.15)  z(F)  ■ z^(F)  whenever  c(F)  £ 

where  Hn  =■  [d(IN2)/ (N+l)  - N(N+2)/(2  lg  3)  ]/[  (N+2)/ (2  lg  3)  - 1]. 

"fc 

which  means  that  the  optimal  information  number  n = 2 . 3 


Proof 


Let  N ■ 1.  Note  that  (5.12)  directly  follows  from  (4.22),  (5.11)  and  the 
f.ict  that  d(I^)  = 7.  We  show  that  z^(F)  s z^(F)  for  any  n*  In  fact 

Z3(F)  * Tg~3  C(F)  + fg"7  S min  y~(c(F)+1)  = 2 (c  (F)+1  min  zn(F). 

n^3  n^3 

This  yields  c(F)  * - 2^/ (2  - ± 22.5  which  holds  due  to  (5.13). 

Hence  z(F)  is  minimized  for  n = 3. 

Let  N *■  2.  The  lefthand  side  of  (5.14)  follows  from  (4.22)  since  Nc  ■ c(F) 

F 

The  righthand  side  of  (5.14)  is  the  complexity  index  of  Newton  iteration.  To 
prove  (5.15)  observe 


z (F)  * (N+l)  [c  (F)  + d(IN  )/(N+l)]  * min 


“2V*  ' — v+*  I */  L W / -T-  uViH*//  V.i’rr-i.y  J — luxii  ; (c  (F)+N) 

n^3  8 n 


/N+n-l\ 

(c 


C1:2)  (c(F)+N)/lg  3 s min  z (F)  , 
7 n*3  n 


which  is  equivalent  to  c(F)  2 This  holds  due  to  (5.15).  Hence  z(F)  is 


minimized  for  n » 2 . 
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Remark  5.2 


If  Gauss  elimination  is  used  for  solving  linear  equations  then  the  com- 

2 3 2 

binatory  cost  dCIN^)  of  Newton  iteration  is  equal  to  d^N^)  = - N + 0(N  ). 

(Note  that  we  count  all  arithmetic  operations.)  Then  in  (5.15)  satisfies 

— \A  ^ ^ - 1 j N — 1.1N.  If  the  Strassen  algorithm  is  used  to  solve  the 
's>  ' a 

linear  equations  then  d(IN2)  = 0(NK)  , 3 = lg  7 = 2.81  and  < 0 for  large  N. 

This  means  the  assumption  c(F)  s H is  not  restrictive  in  this  case.  B 

n 


Theorem  5.2  states  that  the  optimal  information  number  is  achieved  for 
* * 

small  n,  n 3 3 in  the  scalar  case  and  n = 2 in  the  multivariate  case,  when 
the  equal  cost  assumption  holds  and  c(F)  is  reasonably  large.  However,  we 
shall  see  in  Section  6 that  the  optimal  information  number  need  not  be  small 
if  the  equal  cost  assumption  does  not  hold. 

We  know  the  optimal  value  of  n when  F satisfies  the  equal  cost  assumption. 
We  seek  an  iteration  whose  complexity  index  z(<o^)  is  equal  to  z(F).  If  c(F) 
is  large  both  lower  and  upper  bounds  for  z(F)  in  (5.12)  and  (5.14)  are  tight. 
Since  the  righthand  side  of  (5.12)  is  close  to  the  complexity  index  of  IN^ 
and  the  righthand  side  of  (5.14)  is  the  complexity  index  of  Newton  iteration 
, we  see  that  any  iterations  in  the  class  IN^  and  IN^  are  close  to  optimal 
among  all  one-point  iterations  in  the  scalar  and  multivariate  cases,  respec- 
tively. Compare  with  theorem  4.2  in  Kung  and  Traub  [74]  where  the  scalar  case 
is  discussed. 

We  formalize  the  idea  of  optimal  algorithm. 


Definition  5.2 

, , * 

A one-point  algorithm  <p  is  optimal  if  (p  6 *n*>  has  order  n and  has  minimal 


combinatory  cost. 
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Let  be  the  minimal  combinatory  cost  of  combining  {fCx^  ,F'  (x^  ,F"(x  .)  } 

3 * 

to  produce  z^+^  such  that  z^+^  - a =•  O(e^)  in  the  scalar  case.  Let  d^(N)  be 

the  minimal  combinatory  cost  of  combining  [F (x^) ,F'  (x^) } to  produce  z^+^ 

2 

such  that  ||z^+j-ar||  = O(e^)  in  the  multivariate  case.  We  know 

* 

3 * d3  * 7 

N+N^  ^ d„(N)  =*  O(N^)  where  6 = lg  7. 

k 

Recall  that  x^+^  is  a zero  of  the  interpolatory  polynomial  w^(x) . Then 
■ O(e^)  where  p =*  3 for  N * 1 and  p “ 2 for  N s 2.  Thus  d^  is 

★ 

the  complexity  of  approximating  a zero  of  a scalar  quadratic  equation  and  d^(N) 

is  the  complexity  of  solving  a linear  equation  of  order  N. 

★ 

Using  the  iteration  1^  with  an  algorithm  of  minimal  complexity  for  ap- 
proximating a zero  of  a scalar  quadratic  equation  and  Newton  iteration 
★ ★ 

IN^  * I2  with  an  algorithm  of  minimal  complexity  for  solving  of  linear  equations, 
we  get  from  theorem  5.2 


* (IN*)  - z3(F)  = z(F) 
z(IN*)  - z2(F)  =■  z(F) 


for  N * 1 
for  N ^ 2 . 


Thus  we  have 


Theorem  5.3 

* * 
The  iteration  1^  is  optimal  for  the  scalar  case  and  Newton  iteration  I? 

is  optimal  for  the  multivariate  case  among  all  one-point  iterations  whenever 

F satisfies  the  equal  cost  assumption  and  c(F)  2 (as  defined  in  theorem 


5.2)  . 
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Remark  5.2 

Along  the  complexity  dimension  Newton  iteration  is  optimal  in  the  multi- 
variate case.  But  along  the  convergence  dimension  the  iterations  I with 

n 

large  n seem  to  be  more  attractive.  However,  to  preserve  a "type  of  global 
convergence"  of  one  has  to  approximate  a zero  of  a polynomial  operator 
equation  and  this  decreases  the  radius  of  convergence  (see  Traub  and 
Wozniakowski  [76c]) . ■ 

We  have  established  the  informational  optimality  of  IN^  and  IN^  with 
respect  to  the  complexity  index.  Of  course  our  primary  interest  is  in  com- 
plexity. Combining  theorems  5.1  and  5.2  we  obtain 

Theorem  5.4 

If  c(F^  ^ C(F)  for  i ^ 1 and 
(i)  for  N - 1,  G(3)(tD2  * 1 and  5(3)  (2H2  * 1 then 

(t~3  c(f>  + i~3^  (lgt-lglgt)  6 comp(IN3)  s ( ~ c (F)  + UO+t) 

(ii)  for  N i 2,  G(2)(tD  * 1 and  5(2X20*  1 then 

(N+l)  [c  (F)  + d(IF7)/ (N+l)  ](lgt-lglgt)  * comp(IN2)  * (N+l)[c(F)  + d ( IN2)  (N+l)  ] • 

• lg(l+t).  ■ 
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6.  OPTIMAL  INFORMATION  NUMBER  FOR  A SPECIAL  PROBLEM 


If  F does  not  satisfy  the  equal  cost  assumption  then  the  optimal  informa- 
tion number  may  be  large.  For  illustration  we  consider 

the  class  of  problems  of  fixed  dimension  N where  the  cost  c(F)  of  evaluating 
F varies  and  each  piece  of  data  in  F' (x) ,F" (x) , . . . costs  the  same  number  of 
arithmetic  operations  (for  instance  if  F is  an  integral  this  often  holds). 

This  means  that 


(6.1)  c(F(i)) 


for  i ■ 1,2  ... . 


where  c^  ” c^(N)  is  a positive  constant.  Then  the  information  complexity  is 
given  by 


u(F;3tn) 


N+n-l] 
n-1  ) 


- Dcj  + c(F) 


From  (4.19),  (5.8)  and  (5.7)  the  minimal  complexity  index  is  bounded  by 


(6.2) 


c(r> + <"~i>fcy 

lg  n 


c (F)  + 
* :n<F)  s 


lg  n 


lg  n 


for  a positive  constant  K independent  of  N and  n. 
Recall  that  we  define  r.  by 


(6.3)  z(F) 


v(F> 


min  z 
n22 


n 


(F). 


The  minimum  is  achieved  since  zn(F)  tends  to  infinity  with  n.  We  are  interested 

★ 

in  finding  n for  large  c(F).  After  some  tedious  algebraic  manipulations  we 
have 


I 
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Theorem  6 . 1 

(6.4)  Zn*(F)  - N (1+Q(1))  , as  c(F)  - +*  . 

For  every  y > 1 there  exists  = cQ(y)  such  that 
J_  v 

Nv  * N 

(6.5)  c(F)  * n £c(F)  forc(F)  a c . a 

Theorem  6.1  states  that  the  minimal  complexity  index  is  roughly  equal  to 

N c(F)/lg  c(F)  and^the  optimal  information  number  n tends  to  infinity  almost 
N 

linearly  with  c(F)  . Note  that 

Zn(F)  = (1+°^1))  as  C(F>  “ +00 

for  fixed  n.  This  means  that  the  "penalty"  associated  with  using  non-optimal 
fixed  n is  equal  to 

z (F)  , 

(6>6)  i^T?)  = i 18  c(F)(1+o(1)) 

which  tends  to  infinity  with  c(F). 
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